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This  research  analyses  the  properties  of  singular  vectors  of  circnlant  matrices.  In 
particular,  it  proves  that  the  singular  values  of  circnlant  matrices  are  doubly  degen¬ 
erate.  This  results  in  a  V  pattern  in  the  Fourier  transform  of  the  singular  vectors. 
An  additional  structure  in  the  Fourier  space  is  observed  and  proven.  These  results 
provide  a  rigorous  and  firm  foundation  for  the  use  of  the  Fourier  space  structures  in 
the  analysis  of  multipath. 

1  Introduction  and  Background 

This  section  introduces  the  notation  and  describes  the  necessary  background  to  make 
the  discussion  in  the  rest  of  the  report  self  contained  and  easier  to  follow. 

1.1  Eigenvalue/Eigenvector  Decomposition 

Many  problems  in  applied  mathematics,  science  and  engineering  lead  to  the  eigen¬ 
value  -  eigenvector  problem [Strang86]: 

Ax  =  Xx  (1) 

where  A  is  an  n  by  n  given  matrix  and  where  the  unknowns  are  the  eigenvalues  A 
and  the  (nontrivial)  eigenvectors  x.  Eq.  (1)  is  nonlinear,  because  A  multiplies  x.  If 
we  can  discover  A,  then  the  equation  for  x  is  linear [Strang88]: 

(A  —  XI)x  =  0  (2) 

The  key  to  the  problem  is  to  notice: 

1.  The  vector  x  is  in  the  nullspace  of  A  —  XI 
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2.  The  number  A  is  chosen  so  that  A  —  XI  has  a  (nontrivial)  nullspace. 

The  eigenvalue  -  eigenvector  problem  of  Eq.  (1)  is  closely  related  to  the  problem 
of  matrix  diagonalization,  also  called  the  eigenvalue  -  eigenvector  decomposition 
(EVD): 

A  =  XAX-1  (3) 

where  the  diagonal  matrix  A  =  diag( Ai,. . . ,  An)  is  the  eigenvalue  matrix  and  the 
matrix  X  is  the  eigenvector  matrix  whose  columns  are  the  eigenvectors  Xi, ...  ,xn. 

1.1.1  EVD  Existence 

Not  all  matrices  are  diagonalizable.  For  example,  the  matrix: 

'  0  1 
0  0 

is  not  diagonalizable.  This  is  so  because  the  eigenvalue  A  =  0  has  algebraic  multi¬ 
plicity  2  and  geometric  multiplicity  1,  i.e. ,  it  has  only  one  independent  eigenvector 
and  we  cannot  construct  X. 

A  matrix  is  diagonalizable  iff  it  has  n  linearly  independent  eigenvectors,  i.e.,  a 
complete  set  of  eigenvectors.  For  example,  any  matrix  with  distinct  eigenvalues  is 
diagonalizable.  Matrices  who  do  not  have  a  complete  set  of  eigenvectors  are  called 
defective. 

1.1.2  EVD  (Non)uniqueness 

The  eigenvector  matrix  X  is  not  unique,  because  any  (nonzero)  multiple  of  an  eigen¬ 
vector  is  again  an  eigenvector.  Therefore  any  column  of  X  can  be  multiplied  by  a 
(nonzero)  constant  and  produce  a  new  eigenvector  matrix  X.  In  addition,  in  the  case 
of  repeated  eigenvalues,  there  is  even  more  freedom.  Any  (nonzero)  linear  combina¬ 
tion  of  eigenvectors  corresponding  to  the  repeated  eigenvalue  is  again  an  eigenvector 
corresponding  to  that  eigenvalue.  Therefore  such  linear  combinations  of  columns  of 
X  yield  new  eigenvector  matrices  X.  In  the  extreme  example,  where  A  —  I,  the 
identity  matrix,  any  inverible  matrix  X  is  an  eigenvector  matrix. 

1.1.3  Eigenvalue/Eigenvector  Ordering 

The  eigenvalues  on  the  diagonal  of  the  eigenvalue  matrix  A  can  be  ordered  arbitrarily 
as  long  as  the  corresponding  eigenvectors  appear  in  the  same  order  as  columns  of 
the  eigenvector  matrix  X.  In  the  case  of  a  repeated  eigenvalue,  the  order  of  the 
corresponding  eigenvectors  is  arbitrary. 


(4) 
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1.1.4  Complex  Eigenvalues /Eigenvectors 

Eigenvalues  and  eigenvectors  of  real  matrices  may  be  complex.  For  example,  consider 
the  EVD  of  the  rotation  matrix: 


-1 

0 


1  1  1  [  i  0 

—i  i  0  —i 


(5) 


Symmetric  matrices  have  real  eigenvalues  and  eigenvectors,  so  their  EVD  involves 
real  A  and  X. 


1.1.5  Normal  Matrices  and  Their  Eigenvalues/Eigenvectors 

A  matrix  N  is  normal  if  it  commutes  with  its  Hermitian  (complex  conjugate  trans¬ 
pose),  i.e.,  if  NNh  =  NhN.  Normal  matrices  are  exactly  those  which  possess  a 
complete  set  of  orthonormal  eigenvectors.  Therefore  the  EVD  Eq.(3)  has  a  special 
form  for  normal  matrices: 

N  =  UAUh  (6) 

where  the  matrix  U  is  unitary  and  consequently  U^1  =  UH .  Examples  of  normal 

matrices  are  symmetric  matrices,  skew  symmetric  matrices,  Hermitian  and  skew 
Hermitian  matrices,  orthogonal  and  unitary  matrices.  If  N  —  S  is  symmetric,  then 
U  —  Q  is  orthogonal  and  Eq.(6)  becomes: 

5  =  QAQT  (7) 

because  QH  =  QT. 

1.2  Singular  Value/Vector  Decomposition 

Singular  Value/Vector  Decomposition  (SVD)  is  the  generalization  of  the  EVD  of  a 
symmetric  matrix  Eq.(7)  to  cases  where  the  EVD  does  not  exist  and  in  particular 
to  rectangular  matrices.  Any  m  by  n  matrix  A  can  be  factored  into: 

A  =  QiTjQ^  =  (orthogonal)  (diagonal)  (orthogonal)  (8) 

where  we  have  now  two  orthogonal  matrices  Q\  and  Q 2  (not  necessarily  transposes  of 
each  other)  instead  of  one.  The  diagonal  matrix  E  is  called  the  singular  value  matrix 
and  its  entries  ay  are  called  the  singular  values.  Singular  values  are  nonnegative  and 
oq,  02 , ,crr,  where  r  is  the  rank  of  A  are  positive. 

The  orthogonal  matrices  Q 1  and  Q2  are  called  the  left  and  right  singular  vector 
matrices,  respectively,  and  the  columns  are  the  (left  and  right)  singular  vectors. 

The  key  to  working  with  rectangular  matrices  A  is,  almost  always,  to  consider 
ATA  and  AAT.  The  r  nonzero  sigular  values  ai,  cr2, . . . ,  ay  are  the  positive  square 
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roots  of  the  nonzero  eigenvalues  of  both  AT A  and  AAT .  The  columns  of  Q\  ( m 
by  m )  are  eigenvectors  of  AAT  and  the  columns  of  Q2  (n  by  n)  are  eigenvectors  of 
ATA. 

For  positive  definite  matrices,  the  SVD  (Eq.8)  is  identical  to  the  EVD  (Eq.7). 
For  indefinite  matrices,  any  negative  eigenvalues  in  A  become  positive  in  E.  For 
example: 

2  0 1  r 1  0  °  1  r 2  °in  ni 

0  -3  =  0  -1  0  0  3  (9) 

_  o  oj  [0  0  iJL°  oJLu 

For  complex  matrices  E  remains  real,  but  Q±  and  Q2  become  unitary.  Then 
(Eq.8)  becomes: 

A  =  I/iEI/f  =  (unitary)  (diagonal)  (unitary)  (10) 

The  columns  of  Q\  and  Q 2  give  orthonormal  basis  for  all  four  fundamental  sub¬ 
spaces: 

first  r  columns  of  Q\.  column  space  of  A 

last  m-r  columns  of  Q\.  left  nullspace  of  A  ,  . 

first  r  columns  of  Q 2:  row  space  of  A  ^ 

last  n-r  columns  of  Q2'.  nullspace  of  A 

The  SVD  chooses  these  bases  in  an  extremely  special  way.  They  are  more  than  just 
orthonormal.  If  A  multiplies  a  column  of  Q 2,  it  produces  a  multiple  of  a  column  of 
Q 1.  This  comes  directly  from  AQ 2  =  Q iS,  looked  at  a  column  at  a  time. 

The  connections  with  AAT  and  AT A  and  must  hold  if  the  formula  QiSQf1  is 
correct.  That  is  easy  to  see: 

AAt  =  (g1EQ^)(g2STQf)  =  QiEEtQ[  and  similarly  AT A  =  Q2STSQ^ 

(12) 

and  similarly  . 

From  the  first,  Q\  must  be  the  eigenvector  matrix  for  AAT .  The  eigenvalue 
matrix  in  the  middle  is  EET  -  which  is  m  by  m  with  af, ,  a2r  on  the  diagonal. 
From  the  second,  Q2  must  be  the  eigenvector  matrix  for  AT A.  The  diagonal  matrix 
ETE  has  the  same  af , . . . ,  of,  but  it  is  n  by  n. 

To  stress  the  fact  that  A  may  be  orthogonal,  let  us  consider  an  example  with  a 
single  column: 


Here  AT A  is  1  by  1,  while  AAT  is  3  by  3.  They  both  have  eigenvalue  9,  whose 
square  root  is  3  in  E.  The  two  zero  eigenvalues  of  AAT  leave  some  freedom  for  the 
eigenvectors  in  the  second  and  third  columns  of  Q  \ .  There  are  many  choices  which 
keep  the  matrix  Q\  orthogonal. 
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1.2.1  SVD  (Non)uniqueness 

In  general,  the  SVD  is  not  unique.  For  example,  consider  A  which  is  already  or¬ 
thogonal,  i.e.,  A  =  Q.  Then  the  singular  value  matrix  must  be  the  identity  matrix: 
E  —  I.  This  allows  a  lot  of  freedom  for  the  decomposition.  For  example,  A  =  QII 
or  A  =  IIQ  or  even  A  =  {QQ2)IQr^. 


1.2.2  SVD  Existence 


We  prove  the  existence  by  explicit  construction.  For  arbitrary  matrix  A  we  need 
to  find  a  diagonal  matrix  E  and  two  orthogonal  matrices  Qi  and  Q 2  such  that 
A  =  Q^Ql 

Consider  the  n  by  n  matrix  AT A.  Since  it  is  symmetric,  it  has  a  complete  set  of 
orthonormal  eigenvectors  Xi, ...  ,xn  and  correponding  eigenvalues  Ai, . . , ,  An.  Since 
ATA  is  nonnegative  definite,  its  r  nonzero  eigenvalues  are  positive  and  we  can  define: 


o’ j  =  J\j  and  q3  =  — — —  j  =  1, 


CTi 


(14) 


Then,  using  Eq.  (14): 


Q2  =  [xl\x2\.  .  ,\xn]  E  =  diag(ai,a2,,  .,,ar)  Qi  =  [qi\q2\ ■  ■  ■  \qm\  (15) 


It  is  obvious  that  Q 2  is  orthogonal  and  that  E  is  diagonal.  We  need  to  check 
that  Q i  is  orthogonal  and  that  QiT,Q-2  =  A.  Qi  is  orthogonal  because  its  columns 
q%  are  orthonormal: 


fly 


xj  AT  Ax  i 


(J  id  j 


\x{Xj 

- -  =  Oij 

CTiCTj 


Check  that  the  SVD  actually  gives  us  A: 


(16) 


ATAt. 

il  J±iAy  J 

(J  id  j 


XiXiXj 

- -  =  % 

OiGj 


(17) 


To  check  that  Q\LQ\  =  A  we  note  that  the  SVD  can  be  written  as  a  sum  of 
rank  1  transformations: 


QiEQf  =  q\<7\X\  +  q2a 2x2  +  . . .  +  qrarxr  (18) 

To  QiYjQz  =  A  it  is  enough  to  check  that  both  sides  act  the  same  way  on  basis 
vectors.  For  x3  a  basis  vector  in  the  row  space: 

j\.X  ’ 

Q 1 1 SigrnaQ^Xj  =  {q\(J\X\  +  q2a9x 2  +  . . .  +  qrarxr)xj  =  q3a3  =  — -aj  =  Ax~  (19) 
For  a  basis  vector  in  the  nullspace,  both  sides  are  zero.  This  completes  the  proof. 
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1.2.3  SVD  Intuition 

To  summarize,  the  SVD  takes  the  rigth  singular  vectors  (eigenvectors  of  ATA),  scales 
them  by  the  singular  values  and  ”  rotates  them”  into  the  left  singular  vectors: 

A  =  QiTiQ, \  =  (rotate) (scale) (project  onto  AT A  eigenvectors)  (20) 


1.3  Circulant  Matrices 


An  m  by  m  matrix  C  is  called  circulant  if  each  row  of  C  can  be  obtained  from  theh 
previous  row  by  circular  rotation  of  elements,  i.e.,  if  we  shift  each  element  in  the 
i-th  row  over  one  column,  with  the  element  in  the  last  column  being  shifted  back  to 
the  first  column,  we  get  the  (i  +  l)sf  row,  unless  i  =  m,  in  which  case  we  get  the 
first  row.[Moon00]  Thus  the  circulant  matrix  generated  by  (ci,  c2, . . . ,  cm)  is: 


C  = 


Cl 

c2 

C3  • 

■  ■  Cm 

Cm 

Cl 

C2  • 

•  •  Cm—  1 

Cm—  1 

Cm 

Cl  • 

•  •  Cm— 2 

c2 

c3 

c4  . 

Ci 

drc(ci,  c2, . . 

•  i  C-m) 

=  circ(c 

(21) 


Apart  from  the  identity  matrix  /,  the  simplest  circulant  matrix  is  the  permuta¬ 
tion  or  the  shift  matrix: 

n  =  drc(0, 1,  0, . . . ,  0)  (22) 

Any  circulant  matrix  C  can  be  expressed  as  a  polynomial  in  the  shift  matrix  II: 

c  =  cj  +  c2n  +  . . .  +  cm  IT1-1  =  ps{  n)  (23) 

Every  circulant  matrix  C  is  diagonalized  by  the  discrete  Fourier  transform  matrix 


F: 


1 

1 

1 

1 

1 

LV 

ca™-1 

F  = 

1 

LU2 

jKm-2) 

1 

um~ 1 

(24) 


—j2n 

where  lu  =  e~^~ .  This  gives  the  EVD: 

C  =  -FAFh 
n 

with  an  explicit  formula  for  the  eigenvalues: 

A  =  diag(ps(l),p5(uj), . . .  ,p?(a)m_1)) 
We  will  hnd  this  formula  very  useful  for  calculating  the  EVD. 


(25) 

(26) 
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2  The  F 2  Structure 


In  the  analysis  of  multipath  several  structures  have  been  observed  in  the  Fourier 
space  of  the  singular  vectors  of  the  data  matrix[Larue04] .  This  opens  two  questions: 
what  is  the  origin  of  these  structures  and  how  can  they  be  used  to  improve  the 
algorithms  for  the  multipath  detection. 

In  order  to  understand  the  more  complicated  structures  which  appear  in  realistic 
data,  the  simpler  and  generic  data  independent  structures  need  to  be  understood 
first.  We  believe  the  simplest  of  these  structures  is  what  we  call  the  F2  structure 
which  we  describe  and  explain  now.  It  emerges  when  we  study  the  eigenvectors  of 
a  generic  circulant  matrix  C  in  the  Fourier  space. 

The  eigenvectors  of  a  circulant  matrix  C  are  the  columns  of  the  Fourier  matrix  F 
(Eq.??).  They  are  associated  with  a  definite  frequency.  Suppose  we  want  to  analyze 
these  eigenvectors  in  the  Fourier  space.  That  means  taking  their  discrete  Fourier 
transform,  i.e.,  multiplying  them  by  the  DFT  matrix,  which  is  exactly  the  Fourier 
matrix  F.  In  matrix  form,  this  corresponds  to  FF  =  F1 . 

We  want  to  explain  the  structure  which  we  observe  by  plotting  the  DFT  of  the 
eigenvectors,  i.e.,  the  F2.  The  structure  has  a  dot  in  the  (0,0)  corner  and  a  line 
along  the  (0,  n)  to  (n,  0)  diagonal. 

By  direct  calculation: 


(F'% 


n—  1 

£  LOikCOkj  = 


k= 0 


n—  1 

i>« 


k= 0 


k 


j~jy  -  i  {^n)irj  -  i  () 

ui+i  —  1  —  1 


(27) 


as  long  as  a/+J"  ^  1.  Here  we  used  the  summation  formula  for  the  first  n  terms  of  a 
geometric  series  and  the  fact  that  c on  =  1.  Therefore,  the  only  nonzero  elements  of 
F2  are  those  where  a/-1"-7  =  1.  This  can  happen  in  two  cases: 

1.  i  —  j  —  0:  This  explains  the  dot  in  the  (0,0)  corner  of  the  matlab  ’’imagesc” 
picture  of  the  F2  matrix 

2.  i  +  j  =  n:  This  explains  the  straight  line  along  the  (0,  n)  -  (n,  0)  diagonal  of 
F2 


In  both  cases: 


n—  1 


i>'+j 


k 


n—  1 


Ei 


=  n 


(28) 


3  SVD  of  Circulant  Matrices 

This  section  illustrates  intuitively,  on  an  example,  why  the  singular  values  of  a 
circulant  matrix  come  in  pairs. 
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Let 


"1200' 
0  12  0 
L  ~  0  0  12 
.2  0  0  1. 

where  II  is  the  permutation  matrix: 


/  +  2n 


n 


"0100" 
0  0  10 
0  0  0  1 
10  0  0 


Then  CT  =  I  +  nT  =  /  +  n3  and 


CTC  =  CCT  =  5I  +  2S  +  2  S3 


5  2  0  2" 
2  5  2  0 
0  2  5  2 
2  0  2  5 


(29) 


(30) 


(31) 


To  study  the  SVD  C  =  Q\T,Q2,  we  look  at  CTC  and  CCT,  because 

CTC  =  {Q2Y?Ql){QJ2Ql)  =  Q2Y?Ql  (32) 

CCT  =  {QlY>QT2){Q2Y?QTl)  =  Qj:2QTl  (33) 

and  in  general  the  singular  values  of  C  are  the  square  roots  of  the  eigenvalues  of 
CTC  (or  CCT ,  they  are  the  same),  the  singular  vectors  of  C  are  the  eigenvectors  of 
CTC  and  the  left  singular  vectors  of  C  are  the  eigenvectors  of  CCT .  For  the  special 
case  of  a  circulant  matrix,  because  of  CTC  =  CCT ,  the  singular  vectors  and  the  left 
singular  vectors  are  the  same,  i.e.,  Q\  —  Q2- 

So  let  us  compute  the  eigenvalues  of  CTC  by  Moon’s  theorem  \  =  PcTc(a;*)  = 


j2  7T 

5  +  2u/  +  2u~\  where  u  =  e^~  —  —  j: 

A0  =  5  +  2  +  2  =  9  (34) 

A,  =  5  —  2  j  +  2  j  =  5  (35) 

A2  =  5  —  2  —  2  =  1  (36) 

A3  =  5  +  2  j  —  2j  =  5  (37) 


Note  that  Ai  =  A3.  In  general,  it  will  be  true  that  A*  =  A n_j,  because  the  polynomial 
PcTc  is  symmetric  under  %  — >  n  —  i. 

The  CTC  is  diagonalized  by  a  Fourier  matrix  CTC  =  -{FAFH ,  or  explicitly: 
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Note  that  CTC  has  (at  least)  two  diagonalizations,  namely  Eq.  (33)  and  Eq. 
(39).  Since  diagonalization  is  essentially  unique,  i.e.,  up  to  a  change  of  basis,  the 
first  (if  we  start  counting  with  zero)  and  third  column  of  Q2  are  linear  combinations 
of  the  first  and  third  column  of  F.  Since  Q2  is  real,  we  can  simply  take  the  real  and 
imaginary  part  of  the  corresponding  columns  of  F. 

More  explicitly,  consider  the  spectral  decomposition  of  CTC: 


CTC  = 
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The  two  terms  in  parenthesis  are  complex  conjugates  of  each  other,  so  we  can  replace 
them  with  twice  the  real  part  of  the  Erst.  In  addition,  rewrite  the  complex  Fourier 
vector  as  a  sum  of  two  real  waves: 
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This  allows  us  to  rewrite  the  parenthesis  as 
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Finally,  CTC  can  be  rewritten  as: 
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which  checks. 

Singular  values  of  C  are  square  roots  of  the  eigenvalues  of  CTC ,  so: 
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Q2  is  orthogonal,  so  need  to  make  columns  length  1: 
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Note  that  we  reordered  the  singular  vectors  as  we  are  putting  them  into  columns  of 
Q 2  in  order  to  be  aligned  with  the  singular  values,  which  are  ordered  from  largest 
to  smallest. 

To  find  Qi,  compute  CQ2- 
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and  then  divide  the 
orthonormal,  so 


columns  by  the  corresponding  singular  values  to  make  them 
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Check: 
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3.1  The  V  Conjecture 

While  one  example  is  obviously  not  the  proof,  the  hope  is  that  we  can  prove  that 
circulant  matrices  have  singular  values  occuring  in  pairs,  with  the  exception  of  Ao 
and,  for  even  n,  A«.  The  singular  vectors  arc  linear  combinations  of  only  two  Fourier 
vectors.  This  is  what  gives  two  bright  spots  in  the  DFT  of  singular  vectors.  In  cases 
where  the  Fourier  coefficients  decay  monotonically  with  frequency,  we  get  the  ”V” 
pattern. 


3.2  The  W  Conjecture 

In  order  to  get  the  ”‘W’”  pattern,  which  is  empirically  observed  in  many  examples, 
we  need  more  degeneracy,  i.e.,  more  singular  values  the  same.  We  conjecture  that 
the  class  of  circulant  matrices  generated  by  a  sinusoidal  wave  vector  has  the  singular 
values  sigmas  (and  therefore  lambdas)  come  in  groups  of  4.  Furthermore,  the  largest 
singular  values  correspond  to  the  frequencies  of  the  sinusoidal  wave  and  the  rest  of 
the  singular  values  decay  monotonically  with  the  distance  from  this  frequency.  This 
produces  the  structure  shaped  like  the  letter  W. 


4  Conclusion  and  Future  Plans 

This  research  provides  additional  evidence  that  the  singular  vector  structures  in  the 
Fourier  space,  first  observed  and  studied  by  James  Larue [Larue04],  are  robust  and 
relevant  for  the  analysis  of  multipath.  The  existence  of  two  of  these  structures  is 
proven  and  their  origin  explained.  Several  more  structures  have  been  observed  and 
we  believe  they  deserve  further  study.  In  addition,  the  question  of  how  to  best  use 
these  structures  to  detect  multipath  is  a  promising  direction  for  future  research. 
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